Display System and Method

ABSTRACT

A display system and method are disclosed. The system comprises a data transmitting apparatus built in a computer and a data receiving apparatus connected to a remote VGA/DVI display device, in which the data transmitting apparatus comprises a collecting unit for collecting screen data and/or audio data in video signals and a local control unit for encoding said screen data and/or audio data and transmitting said encoded screen data and/or audio data; the data receiving apparatus comprises a remote control unit for controlling the reception of said screen data and/or audio data and decoding it, a buffer unit for controlling the audio/video data decoded by the remote control unit to be placed into a memory for buffering, and an output unit for converting the buffered screen data and/or audio data into a format supported by the VGA/DVI display device and/or audio format.

BACKGROUND OF THE INVENTION

1. Field of Invention

The present application relates to multi-display technology, inparticular to a display system and method capable of supportingmulti-display.

2. Description of Related Prior Art

With the development of various interface techniques, the notebook as aportable computer is required to support a variety of peripheralinterfaces such as USB, serial and parallel port, network interface,VGA/DVI/HDMI interface and Displayport interface in the future.Accordingly, the notebook computer will become complicated and clumsy inorder to support all types of these interfaces.

In view of the above issues, a concept has been proposed in the art thatis based on the port replicator or docking station. That is, aperipheral device is designed as an attachment to the notebook computerand placed near the computer or under the base of the computer in awired way, such as USB/PCIe, or in a wireless way, such as UWB. Allkinds of interfaces can be integrated into the peripheral device, andthus the interface design for the notebook computer can be remarkablysimplified since it needs to support only a few interfaces most commonlyused. In fact, the docking station has been becoming one of the mostimportant attachments in the field of the notebook computer.

FIG. 1 is a schematic diagram of a USB docking station. As shown in FIG.1, one side of the USB docking station is connected to a notebookcomputer via a USB bus, and the other side thereof is coupled to anexternal storage device, such as a SD card, and an image capture device,such as a computer camera. The docking station converts data signalsfrom the notebook computer into a data format adapted to the externalstorage device which stores the data. Also, the acquired image signalsis obtained from the image capture device and transferred to the hostvia USB or UWB. Since other types of interfaces in the docking station,such as parallel port, Ethernet interface and the like, have a low datarate, it is easy to implement conversion from USB/UWB to theseinterfaces. Unfortunately, there has not been any mature solution forUSB/PCIe or UWB to support a second display device, that is, for theconversion from USB/UWB/PCIe to VGA/DVI/Displayport, etc.

According to Patent Reference 1 (US2004117538), the video card in a hostis removed, and display signals generated by OS are directly outputtedto a USB interface. And, the signals are transmitted from the interfaceto another remote module, in which VGA signals are regenerated from thetransmitted signals and outputted directly to a VGA display device.

Patent Reference 2 (US2002135584) discloses a video graphic adapterwhich drives a sub display device of a dual-mode display device with USBinterface. In Patent Reference 2, video signals outputted from USB aredirectly stored in a memory area, and the stored video signals areconverted into analog VGA signal output by a D/A converter.

Furthermore, video needs to be transferred wirelessly to a remoteterminal for display in many situations, for example, the communicationbetween a wireless projector and a notebook computer. In this case, thenotebook computer needs to perform a real-time acquisition of screendata and deliver the data wirelessly to the projector for output. Mostof schemes for wireless display are based on such wireless technology asdefined in IEEE802.11, however. In this way, the high throughputrequired by the display device cannot be well achieved due to thebandwidth of IEEE802.11, and thus it is necessary to employ a deeplylossy compression scheme. This would result in the problem that thedisplayed image cannot resemble and synchronize with the original onevery well, leading to the degrading of screen quality and colordistortion. Meanwhile, the above scheme burdens both of the computer andthe display device with a heavy computational load, and thus affects thesystem performance to a great extent. Moreover, cost tends to beuncontrollable, since the display side usually requires a typicalembedded system including CPU/OS/DSP to retrieve the video and output.In conclusion, because of the limited bandwidth of the existing 802.11system, screen data of high-quality cannot be transmitted timely to thedisplay device side while the computer is playing real-time video,especially video of high-quality.

SUMMARY OF THE INVENTION

The present invention is made in consideration of the above problems,and an object of the present invention is to provide a display systemand method supporting multi-display.

In one aspect of the present invention, a display system is providedcomprising a data transmitting apparatus built in a computer and a datareceiving apparatus connected to a remote VGA/DVI display device, inwhich the data transmitting apparatus comprises a collecting unit forcollecting screen data and/or audio data in video signals and a localcontrol unit for encoding said screen data and/or audio data andtransmitting said encoded screen data and/or audio data; the datareceiving apparatus comprises a remote control unit for controlling thereception of said screen data and/or audio data and decoding it, abuffer unit for controlling the audio/video data decoded by the remotecontrol unit to be placed into a memory for buffering, and an outputunit for converting the buffered screen data and/or audio data into aformat supported by the VGA/DVI display device and/or audio format.

Preferably, the data transmitting apparatus further comprises acompression unit for compressing said screen data and/or audio data byuse of a predetermined compression algorithm, and the data receivingapparatus further comprises a decompression unit for decompressing thecompressed screen data and/or audio data by use of a predetermineddecompression algorithm.

Preferably, the display system supports multi-session function. The datatransmitting apparatus further comprises an input/output mapping unitfor mapping a remote input/output operation into a local input/outputoperation, and the data receiving apparatus further comprises aninput/output interface unit connecting a keyboard/mouse for aninput/output operation by a user.

Preferably, the data receiving apparatus is realized with embedded CPU,FPGA or chip.

Preferably, the screen data and/or audio data are obtained by theoperating system of the host.

Preferably, the data transmitting apparatus captures data for display byvirtualizing a video card device with Mirror Driver under Windows.

Preferably, the screen data and/or audio data are transmitted viaUSB/PCIe interface.

Preferably, the screen data and/or audio data are transmitted in theform of UWB.

In another aspect of the present invention, a method for a displaysystem supporting multi-display comprises steps of collecting screendata and/or audio data from a host, transmitting said screen data and/oraudio data to a remote display device and/or acoustic device, convertingsaid screen data and/or audio data into the supported VGA/DVI format andaudio format and presenting the converted data by the display deviceand/or acoustic device.

Preferably, the method further comprises a step of compressing saidscreen data and/or audio data by use of a predetermined compressionalgorithm before the transmitting step and a step of decompressing thecompressed screen data and/or audio data before the converting step.

Preferably, the method further comprises a step of transmitting acontrol command, and the display device and/or acoustic device receivesaid control command to switch between operating modes.

Preferably, the collecting step comprises obtaining the screen dataand/or audio data by the operating system of the host.

Preferably, data for display is captured by virtualizing a video carddevice with Mirror Driver under Windows.

With the above configuration of the present invention, requirement onbandwidth is lowered since only the dynamically changing part of thescreen data in the video signals or the full-screen data is firstcompressed and then transmitted to the display device side via USBinterface or in the form of UWB. In addition, the present invention canbe realized with a low-cost embedded CPU, FPGA or application-specificchip, with a reduced complexity of the system.

BRIEF DESCRIPTION OF THE DRAWINGS

The above advantages and features of the present invention will be moreapparent from the following detailed description taken conjunction withthe drawings in which:

FIG. 1 shows a schematic diagram of an existing USB docking station;

FIG. 2 shows a block diagram of a display system according to the firstembodiment of the present invention;

FIG. 3 is a schematic diagram for explaining the format of data packetformed during data packetization of the first embodiment;

FIG. 4 shows an example of a data receiving apparatus in the displaysystem of the first embodiment;

FIG. 5 shows a block diagram of the variation of the display systemaccording to the first embodiment of the present invention;

FIG. 6 is a block diagram of a display system according to the secondembodiment of the present invention; and

FIG. 7 is a flowchart diagram for a data transmitting apparatus of thedisplay system according to the second embodiment of the presentinvention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereafter, the preferred embodiments of the present invention will beelaborated with reference to the figures, throughout which the samereference signs denote the same or like components, though shown indifferent figures. For the purpose of clarity and conciseness, a detailaccount of known functions and structures incorporated here will beomitted in case that the subject of the present invention be obscured.

FIG. 2 shows a block diagram of a display system according to the firstembodiment of the present invention.

As shown in FIG. 2, the display system of the first embodiment comprisesa data transmitting apparatus 100 built in a host, say, a notebookcomputer, a VGA/DVI display device 300 and an acoustic device 400located remotely, and a data receiving apparatus 200 connected to thedisplay device 300 and the acoustic device 400 and for receivingmultimedia data from the host side and converting the data into a formatsupported by a VGA/PCI display device.

The data transmitting apparatus 100 includes a video collecting unit 110which collects dynamically changing screen data or full-screen data incurrent video signals, from the host like a notebook computer, a audiocollecting unit 120 which collects audio signals synchronously with thecurrent video signals or separate audio signals from the host to obtainaudio data, a data compression unit 130 which compresses the screen dataand the audio data collected by the video collecting unit 110 and theaudio collecting unit 120 with a predetermined compression algorithm,such as MPEG or JPEG compression algorithm or proprietary algorithm, soas to output compressed data, a packetizing unit 140 which form thecompressed data into data packets of a corresponding format based on apredetermined protocol, a first USB/PCIe controller 160 which transfersthe data packets formed by the packetizing unit 140 via a USB/PCIeinterface, and a transmitting-side controller 150 which controls theabove respective units, such as control on synchronization of audio andvideo data, whether to compress certain data packets as well assignaling interaction, for example, mode switching and the like.

The data receiving apparatus 200 includes a second USB/PCIe controller260 which resembles the first USB/PCIe controller 160 in the datatransmitting apparatus 100 and controls the reception of data packetstransmitted from the data transmitting apparatus 100 via a USB/PCIeinterface, an unpacketizing unit 240 which unpacketizes the data packetsreceived by the USB/PCIe controller 260 according to the same protocolas that adopted in packetizing so as to obtain compressed oruncompressed screen data/audio data, a receiving-side controller 250which controls the respective units in the data receiving apparatus 200,such as control on synchronization of audio and video data, whether tocompress certain data packets as well as signaling interaction, a datadecompression unit 230 which, when the receiving-side controller 250determines that the data packets received currently have beencompressed, decompresses the compressed data with a predetermined datacompression algorithm so as to obtain screen data and audio data, and avideo/audio output unit 210 which converts the screen data into videosignals of VGA format and outputs them to the VGA/DVI/DP display device300 for display, as well as performs D/A conversion on the audio data tooutput to the acoustic device 400.

Besides, the data receiving apparatus 200 includes a buffering unit (notshown) which puts the decoded audio and video data into a memory forbuffering. In this case, the output unit 210 converts the bufferedscreen data into video data of VGA/DVI/DP format to output to theVGA/DVI/DP display device 300.

When the receiving-side controller 250 determines that the data packetsoutputted from the unpacketizing unit 240 is uncompressed, these datapackets are transferred to the buffering unit and then to video/audiooutput unit directly, and the screen data or audio data are convertedinto corresponding display signals and analog audio signals for displayon the VGA/DVI display device 300 and for output from the acousticdevice 400, respectively.

FIG. 3 shows the format of data packets used during data packetizationby the packetizing unit in the display system of the first embodiment.As shown in FIG. 3, ‘Synchro flag’ is used in the initialsynchronization between the data transmitting apparatus 100 and the datareceiving apparatus 200 as well as resynchronization in the case of adesynchronization occurring in data transmission. ‘Type’ indicateswhether the current data packet is a video data packet, an audio datapacket or a control signaling packet, and it also denotes whether thedata packet is compressed or uncompressed. ‘Packet length’ representsthe length of the current data packet. ‘Padding length’ represents thelength of padding information excluding valid data in the data packet.‘Data body’ represents the actual message information and may containcoordinates information, etc., for video. ‘CRC’ is used for checking thedata packet.

As described above, the receiving-side controller 250 in the datareceiving apparatus 200 can determine whether the received data packetis compressed or uncompressed and a video or audio data packet on thebasis of the ‘Type’ field therein. In the case that the data in the datapacket have been compressed, the compressed data is decompressed by thedata decompression unit 230 so as to obtain the corresponding screendata and/or audio data.

In addition, if the video configuration played at the host side has beenchanged, for example, the display mode of the host is changed from1024×768 to 720×480, the transmitting-side controller 150 controls thepacketizing unit 140 to form the configuration information of displaydevice into a control signaling packet, which is in turn transferred tothe data receiving apparatus 200 via the USB interface. Thereceiving-side controller 250 in the data receiving apparatus 200reconfigures the remote display device according to the configurationinformation of display device in the received control signaling packetin preparation for subsequent display.

FIG. 4 shows an instance for embodying the data receiving apparatus inthe display system of the first embodiment.

The conversion module in FIG. 2 can be realized with FPGA/ASIC. As anexample, FIG. 4 shows a block diagram obtained when the conversion fromUSB to VGA is implemented in FPGA manner. As shown in FIG. 4, Cy680001is a USB controller and connected to an FPGA-internal USB module, inwhich operations, such as unpacketizing and decompression of datapacket, are performed. In FPGA, a DDR module is connected to an externalRAM, i.e., MT46v, so as to provide storage space for the operation ofthe USB control module. For instance, the USB control module stores theunpacketized and decompressed screen data and audio data into thestorage space provided by the RAM under the control of the DDR controlmodule, supplies the screen data to the VGA control module to convertthem into video signals of VGA format, and then supplies these signalsto ADV7125 for conversion into analog signals to be displayed on thedisplay device.

FIG. 5 shows a block diagram of the variation of the display systemaccording to the first embodiment of the present invention.

As shown in FIG. 5, in order to achieve a terminal supportingdual-display, the data transmitting side is required to supportmulti-session function and can map, for example, I/O operations of akeyboard/mouse from a remote terminal into local operations. In thisway, the host will treat the display of a remote display device andkeyboard/mouse operations as that of a second local user. Except akeyboard/mouse I/O mapping unit 170 and a multi-session processing drive180, the rest of the components have the same structure as that in thedata transmitting apparatus 100 of the first embodiment, and thus thedetail description will be omitted.

Also, the data receiving apparatus 200 further has a keyboard/mouseinterface 270 for connecting an input means 270 like a keyboard/mouse.As such, after the connection of the input means 270, a user can operateremotely, such as text processing and network browsing, in the samemanner as performed locally. In this case, the receiving-side controller250 transfers via the USB/PCIe bus various inputted commands and controlinformation by the input means 270 to the data transmitting apparatus100 at the host side, where these commands and information are mappedinto local operations by the keyboard/mouse I/O mapping unit 170 andthen sent to the multi-session processing drive 180 for processing.

As can be seen from FIG. 5, except the input means 270, the modifieddata transmitting apparatus 200 has the same structure as that of datatransmitting apparatus 200 of the first embodiment, and thus thedetailed description thereof will not be repeated. Moreover, thereceiving side can be in a variety of forms, such as a docking stationwith a keyboard/mouse plus an external display; it can also be made inthe style of a complete notebook computer without only CPU/OS/harddiskand the like, since all applications are executed at the transmittingside, while this ‘dummy’ notebook computer has only I/O function.

The units in the data transmitting apparatus 100 other than the firstUSB/PICe controller 160 can each be realized in software in theoperating system of the host.

Naturally, it will be appreciated by those skilled in the art that thecompression processing function may not be provided when the datatransmitting apparatus at the host side is realized in the form of adata card of the host, and the description thereof will not be repeated.

FIG. 6 is a block diagram for the second embodiment of a wirelessdisplay system according to the present invention.

As shown in FIG. 6, the data transmitting apparatus 100 built in thehost comprises a data collecting unit 510, a packetizing unit 520, afirst UWB (Ultra Wideband) transceiving unit 530 and a transmitting-sidecontroller 540. The data receiving apparatus 200 connected to a displaydevice and an acoustic device (not shown) comprises a second UWBtransceiving unit 610, a receiving-side controller 620, a unpacketizingunit 630 and an output unit 640.

The host here can be all kinds of computer mainframes, such as themainframe for a PC for a server, a notebook computer, a handheld devicelike a cell phone and PDA, etc.; the display device 300 can be variousdisplay devices, such as the monitor (LCD, CRT or the like) of acomputer, a projector or a television.

In the data transmitting apparatus 100, the data collecting unit 510acquires display data from the host OS. The packetizing unit 520 formsthe display data obtained by the data collecting unit 510 intocorresponding data message in accordance with a protocol for networktransmission. The first UWB transceiving unit 530 transfers the datamessage to the data receiving apparatus 200 wirelessly with UWB.

In the transmitting apparatus 100, the transmitting-side controller 140controls the first UWB transceiving unit 530 as well as performs, forexample, authentication for both sides and negotiation on keys.

In the data receiving apparatus 200, the second UWB transceiving unit610 receives wirelessly the data message from the data transmittingapparatus 100 with UWB. The receiving-side controller 620 controls thesecond UWB transceiving unit 610 as well as performs, for example,authentication for both sides and negotiation on keys. The unpacketizingunit 630 unpacketizes the data message received by the second UWBtransceiving unit 610 into screen data in accordance with the protocol.The output unit 640 transfers the screen data to the display device 300for outputting.

It should be noted that the present embodiment transmits digital pixelsignals with UWB, and in the extreme case the frame rate of transmissionis the frame rate at which the video or any content is played, that is,about 30 frames per second for video, other than transmitting at 60/75frames per second of the refresh frequency of a video card.

On the other hand, in order to display pixel data at the receiving sidein real-time manner, the data receiving apparatus 200 as anotherembodiment has a simple display control mechanism (i.e., the function ofa video card) and can utilize the transmitted pixel data of 25 framesdirectly to perform high-speed refresh processing, for example, at 60/75frames per second.

In the data transmitting apparatus 100 of the second embodiment, all theunits except the first UWB transceiving unit 530 can be realized insoftware of the operating system on the host.

Obviously, it will be appreciated by those skilled in the art that thecompression processing function may not be provided when the datatraffic is not so large.

Now referring to FIG. 7 which is a flowchart of the operation at thehost side taking as an example the processing under Windows.

At step S711, the host is powered on and enters WinLogon.

At step S712, a display process is initiated at WinLogon interface.

WinLogon provides GINA.dll, and different logon modes for Windows can becustomized. The operation of initiating the display process can becompleted by modifying relevant parameters in GINA.

Then, the display process carries out a series of operation as follows.

At step S713, the connection as a TCP client to a display serving as aTCP server is established.

At step S714, after a successful connection, Bridge Driver is started,which is primarily responsible for data processing and transport.

At step S715, Mirror is bundled to a virtual desktop such that the imagedata of the desktop can be obtained.

At step S716, Mirror is started to obtain pixel data. Here, Mirror isshort for Mirror driver which, under Windows, vitualizes a video carddevice to capture the Windows screen. The bundling and starting ofMirror can be fulfilled by Windows GDI.

At step S717, Bridge Driver opens a soft interrupt (22H) by means of IDT(Interrupt Dispatch table), and Mirror conducts an invocation via anassemble instruction int 22H.

At step S718, a transport thread is initiated and an interrupt is openedto obtained information sent out by Mirror. The transport thread takescharge of the transport of image data.

At step S719, Bridge Driver retrieves the information on screen changein response to the interrupt sent by Mirror.

At step S720, the rectangular region of screen change is extracted basedon the information provided by Mirror and added to a list.

At step S721, the list is obtained for subsequent transport processing.

At step S722, a port-reading transport thread is responsible mainly forprocessing related to transport, including data synchronization, messageresponse and the like.

At step S723, according to the obtained list of rectangular regions atstep S721, corresponding data is retrieved and then transferred to thedisplay device.

The solution shown in FIG. 7 has the most impressive advantage in thatwireless display can be implemented as soon as the system enters thestart interface for OS (e.g., Windows login interface) rather thanwaiting until login is finished.

The foregoing description is intended to only illustrate the embodimentsof the present invention. Those skilled in the art will understand thatany modification and partial substitution made within the scope of thepresent invention should be encompassed by the scope of the presentinvention in the claims. Thus, the scope of the present invention shouldbe defined by the appended claims.

1. A display system comprising a data transmitting apparatus built in acomputer and a data receiving apparatus connected to a remote VGA/DVIdisplay device, wherein said data transmitting apparatus comprises: acollecting unit for collecting screen data and/or audio data in videosignals; and a local control unit for encoding said screen data and/oraudio data and transmitting encoded screen data and/or audio data; saiddata receiving apparatus comprises: a remote control unit forcontrolling the reception of said screen data and/or audio data anddecoding; a buffer unit for controlling the audio/video data decoded bythe remote control unit to be placed into a memory for buffering; and anoutput unit for converting the buffered screen data and/or audio datainto a format supported by the VGA/DVI display device and/or audioformat.
 2. The display system of claim 1, wherein said data transmittingapparatus further comprises: a compression unit for compressing saidscreen data and/or audio data by use of a predetermined compressionalgorithm; and said data receiving apparatus further comprises: adecompression unit for decompressing the compressed screen data and/oraudio data by use of a predetermined decompression algorithm.
 3. Thedisplay system of claim 1, wherein said display system supportsmulti-session function; said data transmitting apparatus furthercomprises: an input/output mapping unit for mapping a remoteinput/output operation into a local input/output operation; and saiddata receiving apparatus further comprises: an input/output interfaceunit for connecting a keyboard/mouse for an input/output operation by auser.
 4. The display system of claim 3, wherein said data receivingapparatus is realized with an embedded CPU, FPGA or chip.
 5. The displaysystem of claim 1, wherein the screen data and/or audio data areobtained via the operating system of said computer.
 6. The displaysystem of claim 5, wherein said data transmitting apparatus capturesdata for display by virtualizing a video card device with a MirrorDriver under Windows.
 7. The display system of claim 1, wherein saidscreen data and/or audio data are transmitted via USB/PCIe interface. 8.The display system of claim 1, wherein said screen data and/or audiodata are transmitted in the form of UWB.
 9. A method used in a displaysystem capable of supporting multi-display, comprising the steps of:collecting screen data and/or audio data from a computer; transmittingsaid screen data and/or audio data to a remote display device and/oracoustic device; and converting said screen data and/or audio data intoa supported VGA/DVI format and audio format and presenting the converteddata by the display device and/or acoustic device.
 10. The method ofclaim 9, further comprising: compressing said screen data and/or audiodata by use of a predetermined compression algorithm before saidtransmitting step, and decompressing the compressed screen data and/oraudio data before said converting step.
 11. The method of claim 10,further comprising: transmitting a control command; and the displaydevice and/or acoustic device receiving said control command to switchbetween operating modes.
 12. The method of claim 9, wherein said step ofcollecting comprises obtaining the screen data and/or audio data via theoperating system of said computer.
 13. The method of claim 12, whereindata for display are captured by virtualizing a video card device with aMirror Driver under Windows.
 14. The method of claim 9, wherein saidscreen data and/or audio data are transmitted via USB interface.
 15. Themethod of claim 9, wherein said screen data and/or audio data aretransmitted in the form of UWB.